It has been known to provide streaming content to devices including gaming terminals as disclosed. A potential disadvantage of a streaming architecture is reliance upon the network bandwidth. The network must have an adequate bandwidth to support, for example, concurrent streams to multiple client devices such as gaming terminals, while providing a high degree of reliability. When a network failure occurs between a client device and a streaming server that delivers content, certain functionalities cease. Where the client device is a gaming terminal, such interruption results in lost revenue until the network functionality is restored because the player may not make wagers and play the game. Aside from the monetary losses, particularly in a gaming environment, network failures can have a negative effect on the goodwill and reputation of the streaming content provider such as a casino. The foregoing problem resulting from network failures does not exist with standalone gaming devices that do not rely upon network delivered content and routing player inputs to a remote host. An existing solution to the network failure problem is to invest in a high performance network aiming at 99.999% uptime. Such a solution is expensive and does not solve the problem of a network failure affecting all of the streaming gaming terminals on a casino floor.
To reduce the consumption of network bandwidth, a remote server compresses data files, (e.g., content for delivery) through known streaming protocols such as Real-Time Streaming Protocol (RTSP), MICROSOFT Windows Media HTTP Streaming Protocol (MS-WMSP) and Real-Time Messaging Protocol (RTMP). In a commonly owned U.S. Patent Application Pub. No. 2013/0097220, filed Oct. 14, 2011 as Ser. No. 13/273,555 and entitled “STREAMING BITRATE CONTROL AND MANAGEMENT,” the disclosure of which is incorporated by reference, there is disclosed a technique for managing the compression to improve the consumption of network bandwidth. Even though compression techniques may improve the consumption of network bandwidth, the problem for streaming content remains because network failures occur and reliability issues remain.
The use of virtual machines (“VMs”) is known. A host server may host several VMs, and each VM operates an instance of a game including graphic content. One host server may host multiple VMs accessed by terminals over the network. The use of VMs reduces hardware costs. Where VMs are used it is possible to migrate the interactive game instance from one host server to another host server with minimal downtime. This is also referred to as a “live migration.” Citrix Systems Inc. of Santa Clara, Calif. provides a hypervisor that supports live migration under the name XenMotion®.
The migration solution by CITRIX has several problems causing it unsuitable for a gaming use. The migration solution by CITRIX achieves the downtime of 50-300 ms for a virtual machine to be moved from one host to another. For a streaming gaming machine (or even a streaming video game), any pause in the interactive experience is noticeable to the user. Second, XenMotion® (and the VMWARE equivalent VMotion®) does not support the live migration of content applications that use graphics processing units (“GPUs”). Instead, XenMotion® uses a CPU Memory Management Unit (MMU) feature called “page faults” to assist in the synchronization of memory content between the old host and the new host. A page fault is not necessarily a “fault” in the conventional meaning of the word. Instead, it is a feature that allows an operating system to detect that an application is attempting to access memory that is currently not available. Page faults are often used to provide virtual memory to applications. In Windows XP, for example, each application has access to a memory space of 4 gigabytes but only 1 gigabyte may be physically present. The operating system maintains a table that maps physical memory as needed to applications. If an application is dormant, its memory content may be moved to a disk storage, and the associated pages are marked as such. Any access from the application to the memory on the disk causes a page fault that the operating system handles by returning the memory content from the disk before continuing to execute the application.
Page faults can be used by the Xen hypervisor during the migration of a virtual machine (“VM”) to keep checking pages that have been accessed by an application of the operating system (“OS”) running within the VM. These pages must be sent, and the heuristics are used to limit re-sends and balance the final downtime, when heavily used pages must be sent with both the original and destination VM suspended (i.e., the “stop and copy” phase).
Generally, the hardware functionalities of a GPU are not exposed at a device driver level, therefore the CPU of a host computer is not aware of a GPU page accessible via page faults. The Xen hypervisor does not support live migration of a GPU memory because it cannot determine the areas of the GPU memory that are dynamically changing or remaining static. One way of achieving live migration of a GPU would be to perform synchronization of the entire GPU memory during the “stop and copy” phase of the migration. However, because typical GPUs have as much as 1 gigabyte of data to be synchronized, the synchronization of the entire GPU memory would lead to significant downtime on any conventional network. The present disclosure provides better synchronization performance without affecting the downtime.
The present disclosure is directed toward providing a solution to the above drawbacks and reducing network dependencies in streaming content to numerous gaming terminals, particularly in an environment of a casino.